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This research evaluates the effectiveness of an anti-bullying program. Project 
Ploughshares Puppets for Peace (Woodfine, Lubimiv, & Langlois, 1995). Students in 
grades 3 and 4 (N = 129, 69 boys, 60 girls) from two public elementary schools 
completed a questionnaire on bullying at both pretest/post-test. Although Chi-square 
results showed no significant increase in knowledge or skills to deal with bullying, 
responses to open-ended questions indicated that half the students reported feeling 
more confident in managing bullying. These results suggest that evaluations should 
include student perspectives on the impact of a program that extends beyond specific 
program goals. 
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Cette etude evalue Tefficacite du programme de lutte contre 1' intimidation Puppets for 
Peace de Project Ploughshares (Woodfine, Lubimiv et Langlois, 1995). Des eleves de 3® 
et 4® annees (N = 129, 69 gargons, 60 filles) de deux ecoles primaires publiques ont 
repondu a un questionnaire sur 1' intimidation avant et apres la mise en ceuvre du 
programme. Bien que les resultats du test chi carre n'indiquent aucune amelioration 
notable des connaissances et des competences susceptibles de contrer I'intimidation, 
les reponses aux questions ouvertes mettent en evidence que la moitie des eleves 
disent se sentir plus surs d'eux-memes face a I'intimidation. Ces resultats semblent 
indiquer que les evaluations de programmes de prevention devraient inclure les 
points de vue des eleves sur leur impact, sachant que ces derniers depassent souvent 
les objectifs initiaux. 

Mots cles : intimidation, agression par des pairs, evaluation des programmes. 


School bullying has gained a great deal of attention from researchers and 
school administrators (Camodeca, Goossens, Schuengel, & Terwogt, 
2003); many schools are allocating resources to manage bullying. The 
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purpose of the present study was to evaluate the effectiveness of an anti- 
bullying program. Project Ploughshares Puppets for Peace (P4 program). 

Using quantitative analyses of survey responses and an analysis of 
students' open-ended responses to the program, we investigated 
whether the program achieved its intended goals of increasing student 
awareness of types of bullying and strategies to manage it; how useful 
students found the program; and whether students reported more 
bullying as a result of this program. We examined children's 
perspectives about bullying at three points in time and have both 
characterized and interpreted changes and lack of changes in students' 
views of bullying and their cormections to the puppet program. 

Bullying is commonly defined as repetitive aggression directed at a 
peer who is unable to defend him or herself (Slee, 1995; Smith et al., 1999; 
Slee, 1995). Unlike reciprocal aggression where children exert force 
against each other, bullying is directed from one peer against another 
peer who is unable to stop the aggression. This type of aggression is 
typically categorized according to whether the victim directly or 
indirectly experiences an attack from the aggressor (Olweus, 2001). 
Direct forms include physical and verbal bullying; indirect forms include 
behaviors such as actively isolating an individual from the peer group 
(exclusionary) and spreading rumors. Incidents of bullying often include 
the aggressor, a targeted peer, and bystanders, who play a critical role in 
positively reinforcing bullying behaviors (O'Cormell, Pepler, & Craig; 
1999). Because most students in a school are involved, whether by 
bullying others, being targeted, or witnessing bullying, it is important 
that intervention strategies include an entire school population. 

Researchers have identified a variety of biological and 
environmental factors implicated in bullying. For example, children are 
likely to be victimized if they are anxious and isolated from their peers, 
their parents experience depression and conflict, or their parents use an 
authoritarian parenting style at home (Beran & Violate, 2004; Loeber & 
Dishion, 1983). Also, high rates of community crime are associated with 
severe bullying at school (Espelage, Bosworth, & Simon, 2000). 

Although it is important to study the various contextual factors of 
home, school, and the broader community that are related to bullying, it 
is important not to overlook the extensive research demonstrating that 
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children directly as well as indirectly involved in bullying experience 
numerous functioning difficulties. Several reviews of fhe research on 
bullying summarize a variefy of individual characferisfics of children 
who are bullied. Depression, passivity, and shyness have been identified 
(Beran, in press; Espelage & Swearer; 2003). According fo a mefa-analysis 
conducfed by Hawker and Boulfon (2000), children who are bullied are 
likely fo feel lonely and depressed, and have low self-esfeem. Children 
who bully ofhers also exhibif negafive characferisfics. For example, fhese 
children experience high levels of anger and depression and are af risk 
for engaging in criminal behavior as adulfs (Espelage, Bosworfh, & 
Simon; 2001; Olweus, 1991; Slee, 1995). They may also have little 
empafhy for ofhers (Endresen & Olweus, 2001). In addition, sfudenfs 
who wifness bullying reporf helplessness and vulnerability (Craig & 
Pepler, 1992). Indeed, they may require support similar to children who 
are victimized to feel empowered fo speak ouf againsf bullying. 

Researchers have developed a personality framework fo undersfand 
how bullying occurs. Accordingly, sfudenfs fargef peers who have 
difficulfy coping wifh aggressive overfures. These fargefed children may 
feel highly anxious and afraid, and cry easily. Mahady Wilfon, Craig, 
and Pepler (2000) found fhaf fargefed children may lack coping and 
problem-solving sfrafegies fhaf may increase fhe likelihood of anofher 
attack and lead fo long-ferm negafive developmenfal ouf comes. 
Implications of fhese findings suggesf fhaf bully prevention sfrafegies 
should include skills framing for victimized children. In addition, 
children wifnessing fhe bullying may be afraid fhey could also be 
fargefed and fhus require skills and supporf on how fo provide 
assisfance fo victimized children. Alfhough nof sufficienf alone, a skills 
framing approach is often included as one componenf of school-wide 
anfi-bullying programs (e.g.. Lions Quesf, Second Sfep, and Dare fo 
Care: Bully Proofing Your School). Alfhough school- wide programs are 
now being evaluafed (e.g., Beran & Tufty, in press), it is important to also 
examine specific componenfs of fhese programs such as skills framing on 
sfudenfs' responses fo bullying. 
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PROJECT PLOUGHSHARES PUPPETS FOR PEACE PROGRAM 

The P4 program, which uses puppets and a script developed by 
Woodfine, Lubimiv, and Langlois (1995) at the Friends and Neighbours 
Club of Pembroke, Ontario, employs an extended version of fhe scripf 
(e.g., 30 minufes) fo educafe elemenfary school sfudenfs abouf bullying 
and conflicf resolufion. Using fhree-foof, hand-and-rod puppefs, fwo 
puppefeers enacf a scenario involving direcf and indirecf bullying as 
well as a successful resolution. These behaviors occur among two female 
puppefs and a male puppef friend. Following fhe story, sfudenfs are 
invifed fo identify fhe bullying behaviors shown and discuss four main 
sfrafegies fo manage fhem. The sfrafegies, explained as '4 Foofsfeps' as 
developed by fhe firsf aufhor, include ignoring, saying slop, walking 
away, and getting help. The show, which fakes approximafely 45 
minufes, has been shown fo over 100 schools and communify groups. 
The intention of fhis program is fo demonsfrafe differenf types of 
bullying behaviors fo increase sfudenfs' undersfanding of behaviors fhaf 
exemplify bullying. If also shows various sfrafegies fhaf children who 
are bullied and who wifness bullying can use fo discourage if. 

EVALUATION METHOD 

We evaluafed fhe P4 program fo help inform fhe program sfakeholders 
abouf ifs effectiveness and fo guide ifs developmenf. Because fhe P4 
program had already been planned and implemented in many schools, 
we conducfed our evaluation only af fhe completion sfage of fhe 
program (summafive evaluation). Consisfenf wifh fhe P4 program's fwo 
main goals, we designed fhe evaluation fo examine fhe impacf of fhe P4 
program on sfudenfs' undersfanding of bullying behaviors and anfi- 
bullying sfrafegies. We expected that students in the program would 
learn to identify more types of bullying behaviors and more sfrafegies fo 
manage bullying after viewing fhe puppef show in comparison wifh 
fheir awareness of bullying and sfrafegies before viewing fhe show. We 
also compared fhese oufcomes wifh reporfs from sfudenfs who, af bofh 
fimes fhe questionnaires were adminisfered, had nof seen fhe puppef 
show. Also, fo beffer undersfand fhe impacf of fhe program, we asked 
sfudenfs how useful fhey considered fhe program fo be. 
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A problem noted by evaluators of such programs is the heightened 
sensitization students seem to experience as a result of parficipafing in 
anfi-bullying programs (as discussed in Olweus, 1992; Pepler, Craig, 
Ziegler & Charach, 1994). Thai is, sfudenf reporfs of fhe frequency of 
being bullied may increase as a resulf of gaining a beffer undersfanding 
of fhe nafure of bullying. This higher rafe of reporfing may appear fo 
reduce fhe abilify of evaluafion sfudies, which employ self-reporf 
measures, fo defermine fhe effecfiveness of programs in reducing fhe 
frequency of bullying. In response fo fhis issue in evaluafing fhe P4 
program, we examined fhe sensifizafion effecf on sfudenf reporfs of 
being bullied. We expecfed fhaf sfudenfs would reporf a higher 
frequency of bullying affer, in comparison fo before, parficipafing in fhe 
program because fhey had become more aware of bullying. 

METHOD 

Sample and Procedures 

We included sfudenfs in grades 3 and 4 from fwo public schools in a 
large Midwesfem Canadian cify fhaf had scheduled fhe puppef 
performance. Of fhe 140 sfudenfs asked fo parficipafe, 129 refurned 
signed consenf forms for a response rafe of 92 per cenf, giving us a 
sample of 129 sfudenfs (69 boys, 60 girls). 

Alfhough all sfudenfs viewed fhe puppef performance, fhey filled 
ouf fhe quesfiormaire af differenf fimes. Thaf is, half of fhe sfudenfs (n = 
66) completed the measure before and fhen again affer parficipafing in 
fhe P4 program (intervenfion group). For purposes of comparison, fhe 
ofher half of fhe sfudenfs (n = 63) also complefed fhe quesfiormaire fwice 
(before and af fhe end of a class period) buf before viewing fhe 
performance. The unif of random assignmenf was fhe classroom 
whereby sfudenfs from fhe same classroom were assigned fo eifher fhe 
infervenfion or comparison group. A research assisfanf adminisfered fhe 
quesfiormaire fo each class, and sfudenfs wifhouf a signed consenf form 
worked quiefly af fheir desks. Sfudenfs required approximafely 20 
minufes fo complefe fhe quesfiormaire. 

To furfher defermine whefher fhe P4 program had an impacf on 
sfudenfs, we re-adminisfered fhe quesfiormaires fhree monfhs affer fhe 
sfudenfs complefed fhe firsf sef of quesfiormaires (and affer fhey had all 
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seen the show). The response rate was 100 per cent and thus 129 students 
completed the questionnaire. 

Measures 

We designed the bullying questionnaire to measure students' knowledge 
about and experiences with bullying and to determine their ability to 
accurately identify various behaviors as bullying behaviors. Using a 
student with a gender-neutral name, we developed the first set of ifems 
fo describe seven behaviors fhaf included examples of direcf (e.g., "Lee 
hifs ofher kids who are afraid") and indirecf bullying behaviors (e.g., 
"Lee fells unfrue stories"). Also, in fhis sef of seven ifems, we used fhree 
non-bullying behaviors as comparison ifems (e.g., "Lee is rurming and 
falls down"). We asked sfudenfs fo indicate whefher each behavior was a 
form of bullying by answering "yes" or "no." Because some of fhe ifems 
were not examples of bullying, fhe consistency of all fhe ifems is low. 
For fhis reason, we reporf responses fo each item, rather than their sum. 

In the second set of ifems, we asked sfudenfs fo selecf fhe fypes of 
sfrafegies fhey would use if fhey were bullied, using sfrafegies faughf in 
fhe P4 program (e.g., "I would fell fhe kid fo stop") as well as ofher 
sfrafegies (e.g., "I would use humor"). In addition fo fhese positive 
sfrafegies, we included negative sfrafegies (e.g., "I would hif, kick or 
push fhe kid"). Sfudenfs responded fo each ifem by circling "yes" or 
"no." Again, because of fhe variation in fhe fypes of sfrafegies listed, we 
reporf responses for each sfrafegy. 

We took fhe fhird sef of ifems from fhe bullying subscale of fhe 
Colorado School Climafe Survey (Garrify, Jens, Porfer, Sager, & Shorf- 
Camilli, 2000), a six-item subscale fhaf measures sfudenfs' experiences of 
being bullied in fhe lasf monfh (e.g., "I was hif, pushed, or kicked by 
ofher sfudenfs"). Sfudenfs indicafed fhe frequency fhaf fhese behaviors 
occurred on a five-poinf response scale from "never" to "five or more 
fimes per week." The Alpha coeftidenf for fhe infernal reliabilify of fhis 
subscale in fhe presenf sfudy af prefesf was .78. 

Af fhe fhree-monfh follow-up, we re-adminisfered fhe bullying 
quesfiormaire and included an additional question: "Has fhe puppef 
show changed fhe way fhaf you fhink abouf or deal wifh bullying? 
Please fell how." We designed fhis quesfion fo undersfand fhe program's 
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effectiveness according to the students' perspective, and in their own 
words. 

RESULTS 

The first set of analyses examines the impact of the P4 program on 
students' understanding of bullying behaviors and strategies. The 
second set examines sensitization of students' reports of bullying 
experiences. 

Program impact 

We used a pretest/posttest design to determine changes in students' 
understanding of behaviors that constitute bullying and strategies to 
manage bullying. These changes were examined in both the intervention 
and comparison groups. To determine whether the intervention and 
comparison groups provided similar responses on the individual items 
at pretest, we conducted Chi-square analyses. Because we completed 
multiple comparisons, we used the Bonferonni correction procedure to 
reduce the likelihood that significant differences would emerge because 
of chance. Using a critical p value of .005 for the bully behaviors and .004 
for the strategies, we found no significant differences and concluded that 
students in the two groups held a similar understanding of bullying 
behaviors and strategies (see Table 1). Also, we found no significant 
gender differences in the types of behaviors and strategies reported by 
each group. 

Results in Table 1 also show that before participating in the P4 
program, most students (82% or more) in the intervention group 
identified aggressive behaviors as forms of bullying. Very few (2-4%) 
reported that the non-bullying items, waiting and falling, are forms of 
bullying. In addition to identifying direct forms of bullying such as 
physical and verbal bullying, the majority of students identified non- 
direct bullying behaviors such as gossiping and isolating a peer. 
However, many students (83%) also stated that bullying occurs when 
"Lee hits another student who hits back," a form of aggression that is 
reciprocal and is thus inconsistent with the bullying definition proposed 
by Olweus (2001); Olweus asserts that bullying occurs when a victim is 
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Table 1 

Frequencies of Student Agreement to Bully Behaviors and Strategies 


Administration time 


1 2 

1 2 

1 

3 



Intervention 

Intervention 



vs. 

and 



comparison 

Comparison 



groups 

groups 

Intervention group 

Comparison group 



{n = 66) 

(n = 63) 

X2 

{N = 129) 


Behaviors 

Lee falls 

1 (2%) 

2 (3%) 

1 (2%) 

1 (2%) 

0.00 

3 (2%) 

Lee swears 

65 (98%) 

64 (97%) 

57 (90%) 

55 (89%) 

4.03 

116 (93%) 

Lee hurts 

65 (98%) 

64 (97%) 

56 (89%) 

55 (89%) 

5.10* 

119 (95%) 

others 

Lee waits 

3 (4%) 

4 (6%) 

3 (5%) 

3 (5%) 

0.01 

6 (5%) 

Lee hits 

65 (98%) 

63 (96%) 

55 (87%) 

57 (90%) 

6.21* 

119 (95%) 

Lee talks 

54 (82%) 

56 (85%) 

43 (68%) 

45 (71%) 

3.18 

104 (84%) 

about 

hurting 

others 

Lee tells 

59 (89%) 

61 (92%) 

51 (81%) 

55 (87%) 

1.83 

113 (91%) 

untrue 

stories 

Lee says mean 

65 (98%) 

64 (97%) 

57 (90%) 

58 (92%) 

4.03 

117(94%) 

things 

Lee hits 

55 (83%) 

57 (86%) 

52 (82%) 

52 (82%) 

0.01 

104 (83%) 

someone 

who hits 

back 

Lee gives dirty 

54 (82%) 

55 (83%) 

45 (71%) 

40 (64%) 

1.95 

87 (70%) 

looks 

Strategies 

Get help from 

62 (94%) 

61 (92%) 

60 (97%) 

59 (95%) 

0.58 

119 (95%) 

adult 

Get help from 

46 (70%) 

48 (73%) 

43 (68%) 

41 (66%) 

0.03 

93 (74%) 

kid 

Hit the bully 

0 (0%) 

1 (2%) 

7(11%) 

6 (10%) 

7.75’^* 

3 (2%) 

Say stop 

65 (98%) 

61 (92%) 

59 (95%) 

60 (97%) 

1.17 

120 (96%) 

Ignore the 

51 (77%) 

59 (89%) 

49 (79%) 

49 (79%) 

0.06 

110 (88%) 

bully 

Avoid the 

57 (86%) 

52 (79%) 

58 (92%) 

60 (97%) 

1.08 

105 (84%) 

bully 

Get help from 

52 (79%) 

54 (82%) 

50 (79%) 

49 (79%) 

0.01 

105 (84%) 

parents 

Walk away 

56 (86%) 

59 (89%) 

54 (86%) 

55 (89%) 

0.01 

115 (93%) 

Say mean 

1 (2%) 

2 (3%) 

3 (5%) 

4 (6%) 

1.13 

4 (3%) 
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things 


Use humor 

34 (52%) 

43 (65%) 

23 (36%) 

24 (39%) 

2.94 

67 (54%) 

Positive talk 

56 (85%) 

53 (80%) 

49 (78%) 

46 (77%) 

1.06 

91 (73%) 

Nothing 

5 (8%) 

7 (11%) 

2 (3%) 

6 (10%) 

1.17 

7 (6%) 


Note. ^X^is differences between intervention and comparison groups at first administration of the 
questionnaires. 

*p < .05. **p < .01. 


unable to defend him or herself. Thus, although students identified 
various forms of bullying, fhey were unable to differentiate bullying 
from reciprocal aggression. 

In addifion fo idenfifying several forms of bullying, more fhan half 
of fhe sfudenfs (52%) idenfified numerous posifive sfrafegies fo profecf 
fhemselves from bullying. The mosf commonly reporfed sfrafegies 
included asking for help from an adulf and saying sfop. Few sfudenfs 
reporfed responding wifh aggression because fhey were perhaps 
beginning to adopt an understanding of fhe value and desirabilify of 
responding wifhout fhe use of aggression. Few sfudenfs reporfed doing 
nofhing about the problem of bullying, buf rafher indicafed a variefy of 
sfrafegies fhey would use. 

We then examined how students' understanding of bullying and 
sfrafegies fo deal wifh it changed after seeing the puppet show. We 
compared students' responses before and affer fhe show by using fhe 
McNemar fesf for fwo-relafed samples wifh non-par amefric dafa. In fhis 
analysis, we also compared responses of sfudenfs who had nof seen fhe 
performance af eifher fime of fesfing. We found no significanf differences 
(see Table 1). These resulfs indicafe fhaf fhe number of behaviors fhaf 
sfudenfs idenfified and fhe number of sfrafegies fhey endorsed did nof 
differ significantly between first and second complehon of the 
questionnaire for sfudenfs who participated in the P4 program. Students 
were not better able to differentiate bullying from reciprocal aggression, 
and they did not report using more posihve anh-bullying strategies after 
the puppet show. 

The number of sfudenfs who endorsed each bullying behavior and 
anti-bullying strategy at the three-month follow-up is also shown in 
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Table 1. We examined differences befween boys and girls using Chi- 
square analyses. Using fhe Bonferonni procedure, only one significanf 
difference emerged. More boys (n = 43) fhan girls {n = 24) reporfed fhaf 
fhey would use humor fo cope wifh bullying, X^(l, N = 124) = 8.08, p = 
.004. As shown in fhis fable, a similar percenfage of sfudenfs idenfified 
bullying behaviors and sfrafegies before (3rd column in fable) and affer 
fhe fhree-monfh follow-up (7fh column in fable). 

Responses fo fhe open-ended, wriffen quesfion, "Has fhe puppef 
show changed fhe way fhaf you fhink abouf or deal wifh bullying? 
Please fell how." were coded info five cafegories. Half fhe sfudenfs (50%, 
n = 65) sfafed fhaf fheir fhoughfs or behaviors changed as a resulf of 
seeing fhe puppef show. Their commenfs included, "Yes, because if fold 
me how fo be safer," and "Yes because if showed me how to use the 
strategies, and to try and help others." Less than a third of the students 
(30%, n = 39) felt the show had no impact on them. Their comments 
included, "No, because I've never been bullied," and "No, the puppets' 
problems are too easy to solve." Of the 39 students who stated they were 
not affected, nine (23%) stated that they were already knowledgeable 
about bullying and strategies before seeing the show. This result 
confirms the quantitative results that show that a high percentage of 
students were able to identify behaviors and strategies before seeing the 
show. In addition, 19 students (15%) provided noncommittal responses 
such as "I do not think so. A little, and maybe a little no." Illogical 
responses (3%, n = 4) were unclear statements, and a no response (2%, n = 
2) was coded when students stated they did not remember. 

Responses also varied according to how many ideas were shared. 
Many students who stated that the show had an impact on them also 
provided reasons and ways in which they felt they had been changed; 
whereas, of the students who did not report an impact, most stated no 
without any elaboration. Positive responses included changes in feeling 
(m = 9), thinking (n = 39), and behaving {n = 5). Feeling changes included 
statements about no longer feeling afraid, having more confidence when 
dealing with bullying, and feeling empathy for victims of bullying. 
Changes in how they thought about bullying included being more 
serious about bullying and seeing it as a problem. Behavioral changes 
included statements about helping others, not bullying others, and using 
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several strategies. In summary, these responses suggest that the P4 
program increased students' concern and confidence in managing 
bullying. 

We then compared students who had never been bullied to students 
who had been bullied to determine differences in program effecfs. We 
classified sfudenfs as nof being bullied if fhey indicafed "never" on every 
fype of bullying (on fhe six-ifem bullying scale), and as bullied if fhey 
indicafed "once per week" or more offen on any one or more fypes of 
bullying. Of fhe 28 sfudenfs nof bullied, 16 (57%) sfafed fhaf fhe program 
had a posifive effecf on fhem and 8 (29%) sfafed if did nof. Of fhe 54 
sfudenfs bullied in some way, 24 (44%) sfafed fhe program had a posifive 
effecf and 22 (41%) sfafed if did nof. Thus, sfudenfs who had nof 
experienced bullying found fhe performance fo be more helpful fhan 
sfudenfs who had experienced bullying. Sfudenfs' responses indicafed 
fhaf no differences occurred in how fhe program had an effecf on 
sfudenfs according fo fhe type of bullying fhaf fhey had experienced. 

Self-Reporting Sensitization 

Sfudenfs' responses fo fhe six bullying quesfions are presenfed in Table 
2. To determine whether students reported more bullying after 
participating in the P4 program, we compared their responses at 
administration one and two (before and soon affer fhe show) on fhe sum 
of fhe six-ifem bullying scale, using a paired-samples f-fesf. We found no 
increase in fhe frequency of being bullied when asked before fhe show 
(M = 10.41, SD = 4.27) and again affer fhe show (M = 9.68, SD = 3.68), t{df 
= 64) = 3.78, p < .0001 (fwo-failed). Rafher, a significanf difference 
occurred in fhe opposife direcfion where sfudenfs' reporfs of being 
bullied decreased after seeing the puppet show. As expected, we found 
no significanf difference in reporfs of being bullied befween 
adminisfrafion 1 and 2 in fhe comparison group. 

DISCUSSION 

Because fhe findings of fhe program evaluafion were nof whaf we 
expecfed, we considered fhe dafa in ofher ways and fhoughf abouf new 
ways fo evaluafe bullying programs. The resulfs of fhe presenf sfudy 
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Table 2 


Frequencies of Bullying Behaviors Experienced (N = 129) 


Behaviors 

Never 

Less than 
once/ 
week 

Once/ 

week 

2-4 

times/ 

week 

5 times/ 
week or 

more 

Kids hit me 

79 (62%) 

30 (23%) 

10 (8%) 

8 (6%) 

1 (1%) 

Kids called me 

names 

65 (51%) 

37 (29%) 

14 (11%) 

8 (6%) 

4 (3%) 

Kids told lies 

about me 

79 (61%) 

28 (22%) 

13 (10%) 

5 (4%) 

2 (2%) 

Kids did not let 
me join 

57 (44%) 

44 (34%) 

12 (9%) 

9 (7%) 

5 (4%) 

Kids took my 
things 

92 (71%) 

25 (20%) 

4 (3%) 

3 (2%) 

5 (4%) 

Kids threatened 

me 

99 (78%) 

20 (16%) 

2 (2%) 

4 (3%) 

1 (1%) 


Note. Some items have up to three missing responses. 


indicate that the program did not meet its intended goals of increasing 
awareness and strategies; however students considered the program 
useful in increasing fheir confidence in managing bullying. In addifion, 
rafher fhan becoming sensihzed when reporfing bullying, sfudenfs' 
experiences of bullying seemed fo be normalized affer participating in 
the P4 program. Our results demonstrate that both quantitative results 
and open-ended questions are useful in defermining how anfi-bullying 
programs have an impart on sfudenfs. 

The rafe of bullying reporfed in our sfudy is comparable fo rafes 
reporfed in similar sfudies. Taking fhe average frequency of all fhe 
bullying behaviors, 14 per cenf of sfudenfs reporfed being bullied once 
per week or more offen. Ofher Canadian sfudies have reporfed similar 
rafes, providing evidence of validify for sfudenfs' responses in fhe 
currenf sfudy (Benfley & Li, 1995; Beran & Tuffy, 2002). 

When asked whaf behaviors consfifufe bullying and fhe various 
strafegies fo manage if, sfudenfs demonsfrafed a high level of 
undersfanding even before fhey saw fhe puppef show. If is possible fhaf 
feachers, adminisfrafors, and parenfs are discussing bullying with 
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children, perhaps as a result of heightened media attention on the issue. 
It is also possible that students provided socially desirable responses 
when reporting the types of strategies they would use. These results 
could be confirmed with observational data. Nevertheless, students 
seemed aware of bullying, perhaps because of an increase in demands on 
school administrators and teachers to manage bullying with the 
introduction of anti-bullying initiatives or such policy changes as the 
amendment to the Alberta School Act (1999) to establish safe and caring 
school environments. It is critical, therefore, that developers of anti- 
bullying programs be aware of students' understanding and perceptions 
of bullying before implementing programs, and that this information 
guide the development of these programs. The high initial awareness of 
students in our study reduced the possibility of increasing students' level 
of understanding of behaviors and strategies. This ceiling effect limits 
our determination of how the P4 program had an impact on students. 
Students' ability to differentiate bullying from reciprocal aggression did 
not improve after they had participated in the program. Thus, although 
they could identify bullying behaviors, they considered reciprocal 
aggression (mutual aggression) as a form of bullying (aggression 
targeted at a victim). Program developers must clarify the type of 
behavior and its context that is the focus for intervention. 

Although the P4 program did not have a significant impact on 
students' awareness of bullying and strategies, students indicated they 
experienced an increase in their confidence to deal with bullying and 
their feelings of empathy for children who are bullied. Our findings 
indicate that the program reduced their feelings of fear when dealing 
with children who bully. Considering that many children who are 
bullied or witness bullying feel afraid and intimidated (Craig & Pepler, 
1992; Olweus, 1992), it is unlikely that they will stand up to children who 
bully. Simply providing information about bullying behaviors may not 
be sufficient to increase students' ability to deal with it. However, 
knowledge combined with confidence and concern may provide the 
courage students need to try to stop bullying. According to the students' 
reports, they valued this increase in concern and confidence that they 
gained from the program. Moreover, students who had never been 
bullied indicated that the program was more effective than students who 
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had been bullied. Thus, information and skill-building programs may be 
more beneficial for by-standers than for victimized children, who may 
need more direct support. For these students to gain courage and 
confidence to manage bullying, they likely need ongoing monitoring and 
active support from teachers and students when the bullying actually 
occurs, rather than from simply talking about bullying, as is usually 
done in programs. 

Despite the difficulty of detecting the impact of a program in schools 
already implementing anti-bullying strategies, initial and follow-up 
support about bullying may increase the effectiveness of programs like 
the one evaluated in our study. Indeed, it is unlikely that a 45-minute 
puppet show can significantly change students' long term behaviors, but 
it is plausible that combined with various school-wide initiatives, the 
puppet show may have encouraged students to use strategies to stop 
bullying. 

We had expected that students would become more sensitive to 
reporting bullying if they gained a better understanding of types of 
bullying behaviors. However, student awareness did not increase, the P4 
program did not seem to increase students' reporting of being bullied. 
Instead, the program may have helped students acknowledge that 
bullying occurs, reassure their reactions to bullying, and reduce their 
anxiety. This possibility is supported by students' responses at follow-up 
about how they felt the program made them feel safer when using 
strategies to counter bullying. This impact is important considering that 
many victimized children report high levels of anxiety (Olweus, 1989). 
Thus, when reporting their experiences, students may have under- 
reported rather than over-reported if they felt reassured that bullying 
occurs to other students as well. It is also possible that students gained 
greater confidence in their ability to handle the situation and thus 
minimized their reports of how often bullying occurred. 

Researchers in the future should consider methodological 
limitations. We selected the 45-minute interval between questionnaire 
administrations to correspond with the duration of the puppet show. By 
asking students just before and again after the show about the frequency 
of bullying, we reduced the chances of bullying actually occurring so we 
could determine whether increased reporting was a result of seeing the 
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puppet show rather than experiencing new incidents of bullying. 
However, this short duration may not have been sufficient to affect 
students' responses, and it is possible that students remembered and 
simply repeated their responses from fhe firsf adminisfrafion. This 
design did nof permif an examinafion of whefher fhe number of bullying 
incidenfs decreased as a resulf of the program. Also, the study did not 
measure students' actual use of sfrafegies, buf rafher fheir inclination fo 
use various coping sfrafegies. Anofher limifafion is fhaf children may nof 
have been able fo defecf fhe subfle suggesfion abouf a power imbalance 
in some of fhe examples of bullying behaviors, making if difficulf for 
fhem fo accurafely idenfify bullying. 

In regards fo possible iafrogenic effecfs, fhe puppefeers selecfed 
scenarios fhaf many sfudenfs were mosf likely fo have wifnessed and/or 
experienced, and fhaf would unlikely infroduce new bullying mefhods. 
Af several of fhe performances fhaf I (Tanya) attended, I nofed fhaf 
sfudenfs did nof laugh or appear fo encourage fhe bullying while if was 
being enacfed. Informal feedback from feachers after fhe performances 
indicafed fhaf such scenarios were fypical; fhey menfioned no concerns 
abouf causing sfudenfs furfher disfress. Teachers also fold sfudenfs after 
fhe play fo falk fo someone fhey frusfed abouf how fhey felf during fhe 
puppef show. 

The conclusions of fhis projecf provide one of fhe firsf glimpses of 
changes in sfudenf undersfanding of bullying and sfrafegies for dealing 
wifh bullying following inf ervenf ions. These resulf s also demonsfrafe fhe 
imporfance of using bofh a quanfifafive approach and open-ended 
quesfions. In fhe presenf sfudy, mosf of fhe quanfifafive resulfs were 
nonsignificanf buf sfudenfs' responses fo fhe open-ended question 
revealed fhe program's impacf on sfudenfs' feelings, fhoughfs, and 
behaviors. We recommend fhaf evaluafive approaches include fhe 
mefhods we used in combination wifh alfemafive mefhods fo defermine 
how anfi-bullying programs have an impacf on sfudenfs. For example, 
researchers may ask sfudenfs fo demonsfrafe fheir anfi-bullying 
sfrafegies when read or shown a bullying scenario bofh before and after 
an infervenfion. This type of acfion approach fo fhe idenfificafion of 
sfrafegies would allow sfudenfs to demonstrate the ways they would 
deal with bullying situations prior to and following an educational 
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experience. Thus, a multi-method approach could better determine 
whether the goals of an intervention are met (Rossi, Lipsey, & Freeman, 
2004). In the context of bullying, an approach fhaf encompasses 
quantifafive and qualifafive evidence, and self-reporf and behavior 
measures would help idenfify how sfudenfs sfrucfure fheir 
understandings of managing bullying. If would also assisf program 
developers and facilifafors fo highlighf, reinforce, and add positive 
strategies for sfudenfs fo deal wifh bullying behavior. 

Although the P4 program did not meet the intended goals of 
increasing knowledge and skills, if showed an alfemafive effecf of 
increasing sfudenf courage. More research is needed on how various 
school- and communify-based programs have an impacf on children. 
Very few bully prevention and infervenfion programs have been 
evaluafed and published. Wifh more efforf fo implemenf such programs 
emerging from feachers' concerns abouf how fo manage bullying, if is 
imperative that educators conduct evaluations on this timely issue. This 
study, one of few fo consider fhe effecfs of anti-bullying programs on 
sfudenfs, defermined fhat studenfs seem fo feel empowered and 
confidenf in using anfi-bullying sfrafegies when fhey discussed fhese 
sfrafegies wifh fhe use of puppefs. Because many home, school, and 
individual facfors confribufe fo bullying, a single shorf-ferm sfrafegy 
such as a puppef show can be expecfed fo be useful as only parf of a 
comprehensive, evidence-based, or school- and family-based efforf fo 
help sfudenfs acf in a socially responsible manner. 
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